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(g) Method and apparatus for determining mis-registration. 

(57) An image processing apparatus includes a 
data acquisition stage for acquiring data rep- 
resentative of several spatially separated reg- 
ions of a sample. The apparatus also includes 
data storage means for storing reference data 
corresponding to the sample data. The two sets 
of data are compiled and analyzed to determine 
If the sample is shifted from a nominal position. 
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This invention relates to an electronic image proc- 
essing device which uses automatic image registra- 
tion techniques and equipment for assuring valid reg- 
istration of the images and more particularly to an ap- 
paratus in which several regions are used for the com- 
parison. 

Many image processing steps require two or 
more images to be mutually aligned, based on the 
content of the images. Several methods exist to de- 
termine the amount of mis-registration, and to utilize 
the detected mis-registration to re-register images for 
further processing. The simplest re-registration tech- 
nique is to shift the image data of one or more images 
to align with a single reference image. The amount of 
shift used is detected by the quantitative mis-registra- 
tion detection. 

Quantitative mis-registration detection methods 
currently in use include edge recognition, point recog- 
nition, correlation of row and column sums, correla- 
tion of local regions and others. These methods are 
useful in many situations, but have limitations due to 
accuracy, computation complexity, and limited appli- 
cation in certain image situations. 

Another quantitative mis-registration detection 
method performs a comparison of a test image local 
region with a reference image local region by sum- 
ming the absolute value of the difference of the two 
local regions, on a pixel-by- pixel basis. One of the two 
local regions is then shifted along the X axis by one 
pixel and the summation performed again. This oper- 
ation is repeated a number of times and the summa- 
tion result recorded for each position. The X axis shift 
amount is then returned to the original point and the 
region is shifted along the Y axis by one pixel. The X 
axis shifting and summation is then repeated for this 
new Y axis position. This Y axis shifting, and subse- 
quent X axis shifting and summing is repeated a num- 
ber of times. The recorded summation results are 
then examined and the minimum value noted. The 
minimum summation result corresponds to the pixel 
shift that best aligns the features within the local re- 
gion. 

The amount of shifting is determined by the ex- 
pected maximum mis-registration amount, and is cen- 
tered about the nominal position. The size and posi- 
tion of the local region is selected to include an image 
feature which has significant contrast and non-re- 
peating information content in both the horizontal and 
vertical directions. 

This mis-registration detection method is similar 
in operation to unnormalized correlation of local re- 
gions, but requires much simpler computations as no 
multiplications are needed. 

It is an object of the invention to provide method 
and means to detect mis-registration of test images 
with respect to a reference image. 

Another objective of the present invention is to 
provide a method and apparatus which may readily 



be adapted to several misalignment detection proc- 
esses. 

A limitation of the unnormalized correlation of lo- 
cal regions and the minimum of sums of absolute dif- 
5 ferences of local regions methods is that, for real 
world objects, there may not be a local region with 
sufficient contrast and non-repeating information 
content in both the horizontal and vertical directions. 
In the present invention local region comparison is 
10 performed using two or more local regions from a ref- 
erence image and two or more local regions from the 
test image. The local regions are selected to include 
approximately orthogonal information content in the 
two local regions. The information in the two local re- 
ts gions is spatially linked, but may be widely separated 
within the image frame. To use this concept with the 
sum of absolute differences method, a summation of 
the absolute value of the difference of one of the local 
regions in the reference image and the correspond- 
20 ing local region in the test image, on a pixel- by-pixel 
basis is performed. Then the sum of the absolute dif- 
ferences of the other local region in the reference im- 
age and the corresponding local region in the test im- 
age is performed. These two sums are then added to 
25 each other and the result recorded. The two local re- 
gions in the reference image are then shifted along- 
the X axis by one pixel with respect to the test image 
and the two summations and final sum performed 
again. This operation is repeated a number of times 
30 and the sum of summations recorded for each posi- 
tion. The X axis shift amount is then returned to the 
original point and the regions are shifted along the Y 
axis by one pixel. The X axis shifting and sum of sum- 
mations is then repeated for this new Y axis position. 
35 This Y axis shifting, and subsequent X axis shifting 
and summing is repeated a number of times. The mini- 
mum sum of summations result corresponds to the 
pixel shift that best aligns the features within the two 
local regions. 

40 The concept of using two local regions from a ref- 
erence image to compare against two local regions 
from a test image can be extended to the unnormal- 
ized correlation of local regions method of mis-regis- 
tration determination. The operation is the same as 

45 the previously described two region minimum of sums 
of absolute differences method, except that a sum of 
products is performed at each X and Y axis shift pos- 
ition for each local region, the sums added, and the 
result recorded. When all shifts have been completed, 

so the recorded results are examined and the maximum 
value corresponds to the pixel shift that best aligns 
the features within the two local regions. 

The concept of using two local regions from a ref- 
erence image to compare against two local regions 

55 from a test image using correlation, minimum of sums 
of the absolute differences, or other comparison 
method, can be extended to use any number of local 
regions from a reference image to compare against 
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the same number of local regions from a reference al- 
lows including multiple spatially linked features in the 
comparison, thus ensuring a robust mis-registration 
determination. 

Figure 1 shows an image acquisition system con- 
structed in accordance with this invention; 
Figure 2 shows an object being scanned by the 
system of Figure 1; 

Figure 3 shows several local regions of a refer- 
ence region; and 

Figure 4 shows the elements of the image proc- 
essor of the system of Figure 1 . 
The image acquisition system disclosed herein is 
particularly suited for an apparatus for checking print- 
ed patterns. The system 10 may include for example 
a rotating drum 12 which carries a sheet 14 past a 
scanner 16. Sheet 14 may be imprinted with a plural- 
ity of printed patterns arranged in a rectangular array. 

Scanner 16 is used to scan sheet 14 to produce 
an image of thereof for image acquisition means 18. 
The image of the sheet 14, or at least portions of the 
sheet are loaded in a digital form from means 18 to a 
memory 20. 

Apparatus 10 also includes another memory 22, 
used to store the image of a reference sheet. The two 
images from memories 20 and 22 are provided to an 
image processor 24 which is used to determine if the 
image from scanner 16 has been shifted in one or 
both orthogonal directions. 

Figure 2 shows a printed pattern banknote 26 
from sheet 14 bearing an image 28. In order to deter- 
mine whether the image 28 has been shifted, three 
local areas 30, 32, 34 are selected and analyzed as 
follows. The three local areas may be chosen arbitrar- 
ily, however preferably they are selected to contain 
prominent features extending either in one of two or- 
thogonal directions X or Y. For the purposes of this 
discussion an image 28 is assumed to have been pro- 
duced from a single plate, for example by offset print- 
ing. Frequently, the a printed pattern is printed with 
several superimposed images, each image being 
generated by a different printing method. In such sit- 
uations, the steps presented here are repeated for 
each image so that the orthogonal shift of each im- 
age may be determined. 

Each local image area 30, 32, 34 preferably com- 
prises a two dimensional array of pixels. Memory 22 
may contain either a complete reference image, or lo- 
cal areas of-the reference image corresponding to 
the local areas 30, 32, 34. These local reference 
areas are illustrated in Figure 3 by three pixel arrays 
30', 32', 34'. Advantageously each pixel of the local 
arrays may be represented as a two dimensional co- 
ordinate. For example, the pixel at the upper left hand 
corner of arrays 30', 32', 34' can be referred to as pix- 
els A11\ B11' and C11' respectively. Originally, it is 
assumed that the local areas of the actual image 28 
coincide exactly with the local areas of the reference 



image. In other words, it is assumed that center pixel 
of region 30' (shaded in Figure 3) coincides with the 
center pixel of region 30. 

As shown in more detail in Figure 4, the image 

5 processor 24 includes five counters 40, 42, 44, 46, 48, 
a summer network 50, a memory 52, a comparator 54 
and a shift selector 56. It should be understood that 
the image processor of Figure 4 is preferably imple- 
mented as a digital computer and the components 

10 shown in the Figure are shown for illustrative purpos- 
es. 

In order to determine if the actual image from 
memory 20 is shifted orthogonally with respect to the 
reference image of memory 22, a reiterative process 

is is performed as follows. Counter 40 sequentially des- 
ignates one of the local regions 30', 32' 34' of the ref- 
erence image to the summer network 50. Initially 
counters 42, 44 are set to zero indicating that no shift 
has yet occurred. Counters 46 and 48 are used as 

20 pointers to indicate to the network 50 which pixel of 
an array is to be processed. After the counter 40 des- 
ignates the first local region 30', the summer network 
50 obtains the values of each pixel of region 30 and 
takes the absolute value of the difference of the value 

25 of each pixel of array 30 and array 30'. Thus, t he sum- 
ming network 50 calculates the differences I A11- 
A11'|, |A12-A12'| and so on. These differences are 
summed and stored. The first position of a results ar- 
ray in memory 52 and in a "best match" location also 

30 in memory 52. When the differences for all the pixels 
of arrays 30 and 30' have been calculated and stored, 
counter 40 is incremented and the whole process is 
repeated for arrays 32, 32', with the difference from 
each pixel being summed with the difference of the 

35 calculation for the same position for arrays 30, 30'. In 
other words, for the second set of arrays 32, 32', the 
summing network 50 first obtains the difference B11- 
B11' and adds this difference to the difference A11- 
A11'. The same process is repeated for each of the 

40 other pixels of arrays 32, 32' under the direction of 
counters 46, 48. Finally, the same process is again re- 
peated forthe third set of arrays 34, 34'. Once the cal- 
culations are completed, counter 42 is incremented 
by one to indicate that the reference arrays 30', 

45 32' and 34' must be offset along the x-axis by one pix- 
el. The summer network 50 again performs the 
above-described process, but this time the pixels of 
the reference image are offset so that for example the 
difference A 12- A1V is calculated and stored tempor- 

so arily until the calculations for all the three sets of ar- 
rays are completed. Comparator 54 then compares 
the resulting value with the value resulting with no 
shift This value is placed in the second position in the 
results array in memory 52. If the new results meet 

55 certain criteria (i.e. they are equal to or smaller than 
the old results) the old value in the best match loca- 
tion in memory 52 is replaced with the new ones. If 
the results are replaced, the shifts in the X and Y dh 
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rection as indicated by counters 42, 44 are also stored 
in memory 52. The arrays 30' 32' 34' are then shifted 
again and the whole process is repeated. The three 
arrays 30' 32' 34* are shifted a predetermined num- 
ber of times to t he left and right oft he original position. 
Thereafter, the arrays are also shifted up and down 
a preselected number of times. 

When all the shifts and calculations are complet- 
ed, memory 52 contains the results array shoving the 
values due to the orthogonal shrftings, and the best 
match value and the amount of shift in the X and Y 
direction which generated this result This information 
is provided to shift selector 56 which generates a cor- 
responding X- and Y- shift indicating the orthogonal 
offset of image 28 from a reference image. The results 
array of summations in memory 52 may be used as 
a quantitative measure which may be used to deter- 
mine the level of confidence in the process. 

In the embodiment described above, three local 
areas are used for illustrative purposes. It should be 
understood that any number of such areas may be 
used higher than one. Moreover, other criteria for 
comparing the image and reference arrays without 
departing from the scope of the invention as defined 
in the appended claims. 



Claims 

1. An image processing apparatus comprising: 

image acquisition means for acquiring 
data corresponding to an image, said image in- 
cluding several spatially separated local regions; 

storage means for storing reference infor- 
mation; and 

compiling means for compiling information 
from said local regions and said reference infor- 
mation; 

analyzing means for analyzing the com- 
piled information to determine if said image is 
shifted from a nominal position. 



ence arrays of pixels; 

comparing means for comparing said sam- 
ple and said reference arrays; 

compiling means for compiling the results 
5 of the comparison; and 

analyzing means for determining shifts 
between said sample and reference arrays. 

5. The system of claim 4 wherein said apparatus fur- 
to ther includes shifting means for shifting one of 

said references arrays and said sample arrays in 
a predetermined direction. 

6. Amethod of determining a linear shift of a sample 
is from a nominal position comprising the steps of: 

acquiring image data arranged in several 
sample arrays of pixels, each array correspond- 
ing to a local region of said sample; 

comparing said sample arrays to corre- 
20 sponding reference arrays; 

storing a difference array corresponding 
to the difference of said sample and reference ar- 
rays; 

shifting said sample arrays; and 
25 comparing the difference between said 

shifted sample arrays and said reference arrays 
to said stored difference. 

7. The method of claim 6 wherein said sample and 
30 reference arrays are analyzed on a pixel-by-pixel 

basis. 
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2. The apparatus of claim 1 wherein said informa- 
tion from said local regions and said reference in- 
formation are arranged in a two dimensional ar- 45 
ray of pixels. 

3. The apparatus of claim 2 further comprising shift- 
ing means for shifting information from said local 
regions selectively in one of two orthogonal di- so 
rections. 



4. An image processing apparatus comprising: 

image acquisition means for acquiring 
data from several local regions of a sample, said 55 
data being partitioned into corresponding sample 
arrays of pixels; 

memory means for storing several refer- 
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